Remarks 

Prior to this Amendment, claims 1-30 were pending in this Application. 
In the USPTO Action, the following points were asserted: 

(1) Claims 1 and 16 were rejected under 35 U.S.C. 102(a) over U.S. Patent No, 
6,525,731 ("Suits"); 

(2) Claims 2 and 17 were rejected under 35 U.S.C. 103(a) over Suits and U.S. Patent 
No. 6,570,578 ("Smirnov 1 '); 

(3) Claims 3-7 and 18-22 were rejected under 35 U.S.C. 103(a) over Suits, Smirnov 
and U.S. Patent No. 6,538,654 ("Rose"); 

(4) Claims 8-13 and 23-28 were rejected under 35 U.S.C. 103(a) over Suits, Smirnov, 
and U S. Patent No. 6,266,053 ("French"); and 

(5) Claims 14, 15, 29, and 30 were rejected under 35 U.S.C. 103(a) over Suits, 
Smirnov, French and U.S. Patent No. 6,232,974 ("Horvitz"). 

The claims have been amended herein as follows: Claims 1-5 and 16-20 have been 
canceled without prejudice to re-filing such claims, whether in a Continuation application or 
otherwise. Claims 6, 8, 21 and 23 have been amended. New claims 31 and 32 have been 
added. 

No admission of any kind, whether as to prior art or otherwise, is made hereby. 
Support for the amendment is found in the application as filed, and no new matter has been 
added. 

The prior art rejections, and the differences between the amended claims and the cited 
references, will next be addressed in detail, following a discussion of the claimed invention. 

The Claimed Invention : 

The present invention relates to the field of computer image synthesis. As set forth in 
the specification and drawings, aspects of the present invention provide systems and 
techniques for enabling a network end user to view, on a suitable client device, a two- 
dimensional image that has been rendered from a three-dimensional scene by a server 
computer. Based upon customization inputs received from the end user, the server computer 
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modifies the underlying scene data and uses the modified scene data to re-render the image. 
The re-rendered image is then transmitted back to the end user. 

In order to fully understand the present invention, one must understand the difference 
between "scene data" and an "image," "Scene data" is three-dimensional data including, 
among other things, geometrical models of the pbject(s) contained in the scene. An "image" 
is a two-dimensional rendering of the scene data, based upon a particular viewpoint and 
lighting source(s). Figuratively speaking, scene data can be thought of a collection of three- 
dimensional objects to be photographed, and an image can be thought of as a single 
photograph of those objects. 

These aspects have particular significance in the context of modern networked 
computing architectures. More particularly, as noted in the Background of the Invention (p. 1 
et seq. of the application as filed), Web pages are typically static in nature. In those limited 
situations (such as those using algorithms for three-dimensional viewing) in which non-static 
displays are possible, enormous quantities of data need to be accessed, retrieved and sent, 
which would be prohibitively slow or resource/bandwidth-intensive in the general case. 
Moreover, as stated in the Background section: 

"Even if the bandwidth were sufficient to enable sufficient three-dimensional scene 
information to be retrieved within a reasonable amount of time to facilitate display of the 
scene with a more photo-realistic quality, in a number of situations it may be undesirable to 
transfer the information to the user. For example, if a manufacturer uses the Web site to 
provide information about its products for, for example, potential customers, it may not wish 
to make information sufficient to provide photo-realistic three-dimensional images available 
for retrieval, since information that is sufficiently detailed to generate such images may also 
be sufficiently detailed to provide a significant amount of design information that may be of 
interest to competitors. This is particularly the case if the information is sufficiently detailed 
to allow a user to modify or customize the scene. For example, if the manufacturer is an 
automobile manufacturer, it may be desirable to allow a user to not only view the automobile 
from user-selectable orientation, but also to modify or customize the scene, by, for example, 
changing the color and texture of various surfaces, changing the positions of light sources, 
enable the automobile to be displayed with doors, hood and/or trunk in an open position, and 
the like. The amount of information that would be necessary to allow a user to perform such 
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operations may require a significant amount of time to transfer. In addition, the amount of 
information that may be required may constitute a significant amount of the design 
information for the object(s) in the scene, which may be confidential." 

Thus, as the Background states, "it will be desirable to maintain the three-dimensional 
scene information on the Web site and have the Web site render two-dimensional images in 
orientations and with modifications and customizations of the scene as specified by the user, 
and transmit the two dimensional image information to the user's browser for display. 
However, problems arise since not only will the Web site need to retrieve the information 
from databases on which the information is stored for transmission to the user's browser, but 
need also render the two* dimensional images from orientations and with modifications and 
customizations specified by the user. For example, if a number of users are accessing the Web 
site concurrently, the amount of processing power required to render the images in a 
reasonable amount of time can become quite large, in addition, problems can arise if a group 
of users are making use of the same scene, for whom customizations made by any of the users 
in the group are to be incorporated into the scene as used by all of the members of the group, 
since all of the customizations would need to be transmitted to all of the users and 
incorporated into their respective three-dimensional scenes." 

The claimed invention provides a solution to the above-noted problems of the prior 
art. Among other advantages, the invention enables modifications and customizations in 
response to user requests, without requiring prohibitive amounts of bandwidth and processing 
resources, with the requirement of transmitting customizations to all users, and while enabling 
the information provider to maintain control over the amount of information provided. 

In a system according to the present invention, an end user is shown a two- 
dimensional image of a three-dimensional scene. The user "customizes" the scene by making 
interactive inputs with respect to the image displayed on a local device. These inputs are then 
transmitted to the server, which makes corresponding changes to the scene data and then re- 
renders the two-dimensional image. The re-rendered image is then transmitted back to the 
end user. 

Key features of the invention include, but are not limited, to the following: 
1 . Rendering and re-rendering is performed at the server level . 
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2. Scene data is not sent to the client. Thus, an end user can be provided with the 
ability to customize a scene without providing the end user with access to any proprietary or 
sensitive scene data. 

3. At the client level, the end user works only with an image, using a suitable 
local interface. The end user never works with scene data. This feature may be useful, for 
example, in the case of a computer artist who may be familiar with the manipulation of a 
displayed image, but who may know nothing about the manipulation of scene data. 

4. Because the client only works with image data and performs no rendering, the 
client device does not require a significant amount of computing power. For example, the 
client device can be a small, handheld PDA, "smart" phone, or like device. 

5. Customization are performed by customizing the underlying scene data. 
Therefore, the customizations are now centrally available to other users on the network. 

6. Because customizations are performed on scene data at the server level, the 
information provider can limit, or otherwise control, the customizations that can be performed 
by an end user at the client level. 

7. Further, end users and clients can be assigned to different classes, and can be 
granted varying levels of permissions with respect to customizations of scene data. 

These and other aspects of the invention are reflected in the claims. For example, 
amended claim 6 is directed to a server connected into a network including at least one client 
and a communication link interconnecting the client and server. The server comprises an 
image rendering module configured to render, from three-dimensional scene data representing 
a scene, a two-dimensional image. An interface is configured to transmit the two-dimensional 
image over the communication link to the client. A user interaction control module 
configured to control interactions with the at least one client in connection with rendering of 
the image from the scene data. 

Also in the amended claims, such as claim 6, the control interactions include requests 
from the at least one client. These requests include scene customization information 
requesting at least one customization to the scene. The user interaction control module is 
configured to enable the image rendering module to render an image of the scene as 
customized in relation to the customization information. 
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As will next be discussed in detail, the cited art does not teach or suggest the above 
described aspects of the present invention, and the claims are thus respectfully submitted to be 
patentable over the cited art. 

Rejection of Claims 1 and 16 under 35 U.S.C. 102(a) 

As noted above. Claims 1 and 16 were rejected under 35 U.S.C. 102(a) over Suits. 
Although the Applicant respectfully disagrees with this finding of the Action, the present 
Amendment, in order to move this long-pending application toward allowance, cancels claims 

1 and 16 without prejudice to re-filing such claims, whether in a Continuation application or 
otherwise. 

Rejection of Claims 2 and 17 under 35 U.S.C. 103(a) 

Claims 2 and 17 were rejected under 35 U.S.C. 103(a) over Suits and Smimov. 
Although the Applicant respectfully disagrees with this finding of the Action, the present 
Amendment, in order to move this long-pending application toward allowance, cancels claims 

2 and 17 without prejudice to re-filing such claims, whether in a Continuation or otherwise. 

Rejection of Claims 3-7 and 18-22 under 35 U.S.C. 103(a) 

Claims 3-7 and 18-22 were rejected under 35 U.S.C. 103(a) over a combination of 
three references: Suits, Smirnov and Rose. Although the Applicant respectfully disagrees 
with this finding of the Action, the present Amendment, in order to move this long-pending 
application toward allowance, cancels claims 3-5 and 18-20 without prejudice to re-filing 
such claims, whether in a Continuation or otherwise. 

The allowability of claims 6-7 and 21-22 over Suits, Smirnov and Rose will next be 
addressed. 

Suits purports to disclose a system for providing interactive views of 3-dimensional 
models with surface properties. Suits asserts that the system provides "a compact 
representation of a 3D model and its surface features" and that it allegedly provides for 
efficiently viewing and interacting with the model using dynamically switched texture maps. 
The compact representation, asserts Suits, is beneficial for transmission of the model across a 
network as well as for local storage of the model in the computer memory. The dynamically 
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switched texture maps allow for more accurate surface details on the 3D model, as well as 
speedy interaction between a user and the 3D model. 

Smirnov purports to provide a computer-based system that automates the production 
of so-called pass-images ("passes") from data defining 3-D scenes. For example, a "beauty 
pass" of a selected object shows the normal surface appearance without any background or 
other objects. A shadow pass shows only selected shadows and a highlight pass, only selected 
highlights. The passes allow pixel-processing to be done efficiently. The processed passes 
can then be combined ("composited") to form the finished image. In the past, these passes 
were produced by editing a copy of the 3-D scene for each pass and rendering the edited copy. 
Smirnov asserts that the invention automates the production of passes by filtering the 3-D 
scene through pre-specified pass definitions that override properties of the 3-D scenes. The 
results of filtering are rendered (rendering largely comprises the process of ray-tracing) to 
form the passes. The system stores numerous pass definitions. Each time the 3-D scene is 
edited, the passes can be produced automatically from the pass definitions. This automation of 
pass production also allows the passes to be used in the authoring environment by allowing a 
pass preview of the 3-D scene rendering. 

Rose purports to disclose a system and method for facilitating the production, 
processing, and Internet distribution of 3D animated movies including 3D multipath movies. 
Rose asserts that a key reduction algorithm reduces the keyframes associated with the 3D 
movie to facilitate the streaming of the data over the Internet. An animation optimization and 
texture optimization algorithm allows the system to get statistical information of the portions 
of the 3D object which are invisible (outside the view frame), and whose animation and 
texture data may thus be safely removed. If the 3D object is within the view frame, the 
optimization algorithm gathers information about the distance and size of the 3 D object and 
its associated texture data within the frame. The system and method further allows creation of 
various versions of the 3D object from a single production process. The various versions are 
tagged with an identifier identifying the target platforms and media platforms in which they 
are suitable for display. During playback, the correct version of the 3D object is selected for 
display based on the detected Internet bandwidth and/or CPU speed. To stream the 3D 
animation content over the Internet for real-time playback, the present system and method 
allows the data to be divided into an upfront file and various stream files. The upfront file is 
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downloaded prior to the playback of the movie. The stream files are streamed over the 
Internet and made available to the 3D engine in time for their use. 

However, Claims 6-7 and 21-22, as amended, recite combinations of features not 
taught or suggested by the art of record, and that are patentable over the cited references, 
taken separately or in combination. 

Specifically, the cited references, either separately or in combination, contain no 
teaching or suggestion of the entire combination of features recited by independent claims 6 
and 21, including: a client-server architecture (and a computer program product enabling the 
client-server architecture), that enable a user to interactively select and customize a scene (i.e., 
update the underlying scene data) based upon a rendered image of the scene displayed on a 
client device, where the image has been rendered at the server level, and where image data, 
rather than scene data, has been transmitted to the client device. After a user has customized 
a scene at the client level, the customized scene is re-rendered at the server level. 

The cited references are quite different and distinguishable. In Suits, a server sends 
model data to a client. The client then renders the model data to create an image to be 
displayed. As discussed above, in contrast, in the present invention, no rendering is 
performed at the client level. 

In Smirnov, pre-specified pass data is used to create multiple image layers. The user 
cannot interactively customize the scene. Rather, the user's interaction is limited to 
compositing the rendered passes, without any customization of the underlying scene data and 
re-rendering. Again, this is in direct contrast to the claimed invention. 

Rose describes client-side rendering, where the server's contribution consists of 
selecting which parts of the scene to send to the client. 

Each of these approaches, of Suits, Smirnov and Rose, is fundamentally different from 
the approach set forth in amended claims 6 and 21 . Claims 6 and 21 recite combinations of 
features that is neither taught nor suggested by the cited references, taken separately or in 
combination, and are thus respectfully asserted to be patentable over the references. 

Claim 7 depends from claim 6, and claim 22 depends from claim 21, incorporating all 
of the limitations thereof and adding further limitations thereto. It is therefore asserted that 
claims 7 and 22 are allowable for the reasons advanced above in support of the patentability 
of claims 6 and 21, and further on the basis of the additional limitations therein. 
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For the above reasons, it is respectfully submitted that claims 6-7 and 21-22 are 
allowable over Suits, Smirnov and Rose. 

Rejection of Claims 8-13 and 23-28 under 35 U.S.C. 103(a) 

The Action rejected claims 8-13 and 23-28 under 35 U.S.C. 103(a) as being allegedly 
unpatentable over a combination of Suits, Smirnov, and French, citing, inter alia, cols. 9, 10 
and 12 of Suits, cols. 3, 9, 10, 12 and 13 of Smirnov), and FIG. 2 and cols. 5, 6, 7 and 9 of 
French. 

French purports to disclose a time inheritance scene graph technique, i.e., a technique 
for representing a visual scene as a directed acyclic graph of data and operators that generates 
a sequence of image frames over specified time intervals. The graph specifies temporal and 
spatial values for associated visual elements of the scene. Time is modeled in the inheritance 
properties explicitly defined within the scene graph hierarchy, by assigning temporal 
attributes to each media element Branch nodes of the graph specify transforms for the 
temporal and spatial coordinate systems. To evaluate the appearance or behavior of the scene 
and in particular the global time values of particular elements at a given time instant, the 
graph is traversed in a direction from a root node down toward the leaf nodes, thereby causing 
temporal transformations specified along the branches of the graph to modify time parameters 
of the scene data at the nodes. Chil d nodes are preferably evaluated after being transformed, 
to determine the extent to which they contribute the data to the final scene. Temporal 
transformations may include translation operations that offset temporal event times; scaling 
operations that change the rate at which time passes; or clipping operations, that restrict the 
range of time parameters to exclude the evaluation of parts of the graph. 

However, in the context of the claimed invention, French fails to add anything of 
significance to the other art of record. Specifically, French fails to teach or suggest, either 
singly, or in combination with the other art of record, a client-server architecture, and a 
computer program product enabling the client-server architecture, that allow a user to 
interactively select and customize a scene (i.e., update the underlying scene data) based upon 
a rendered image of the scene displayed on a client device, where the image has been 
rendered at the server level, and where image data, rather than scene data has been 
transmitted to the client device. 
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Claims 8 and 23 depend, respectively, from claims 6 and 21, incorporating all of the 
limitations thereof and adding further limitations thereto. It is thus asserted that claims 8 and 
23 are allowable over the cited art for the reasons advanced above in support of the 
patentability of claims 6 and 21 , and further on the basis of the added limitations. 

Claims 9-13 and 24-28 depend, respectively, from claims 8 and 23, incorporating all 
of the limitations thereof and adding further limitations thereto. It is therefore asserted that 
claims 9-13 and 24-28 are allowable for the reasons advanced above in support of the 
patentability of claims 8 and 23 and further on the basis of the added limitations. 

For the above reasons, it is respectfully asserted that claims 8-13 and 23-28, as 
amended, are allowable over Suits, Smirnov, and French. 

Rejection of Claims 14. IS, 29. and 30 under 35 U.S.C. 103faV 

The Action rejected claims 14, 15, 29 and 30 under 35 U.S.C. 103(a) as being 
allegedly unpatentable over a combination of Suits, Smirnov, French and Horvitz et al. (U.S. 
6,232,974), citing, inter alia, the Abstract and FIG. 9 of Horvitz. 

Horvitz purports to disclose a decision-theoretic regulator that employs a method for 
allocating computational resources to components of media content to create the highest 
quality output for a budget of rendering resources. The components of the content represent 
parts of the content that have independent quality parameters that the regulator can vary to 
trade-off quality for computational savings. For example, in multimedia content, the 
components might be objects in a 3D graphics scene. Horvitz states that the method allocates 
computational resources by attempting to minimize the total expected cost of a rendering task. 
The method computes the raw error for a rendering action on a component and then maps the 
raw error to a perceived error based on empirical evidence of how users perceive errors in 
rendered output. The expected cost is computed from the perceived error or raw error by 
applying a model of attention that gives the probability that a user is focusing his or her 
attention on a component. The method minimizes the total expected cost by selecting a 
rendering action for each component that yields the lowest expected cost for a given rendering 
budget. 

However, in the context of the claimed invention, Horvitz fails to add anything of 
significance to the other art of record. Specifically, Horvitz fails to teach or suggest, either 
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singly, or in combination with the other art of record, a client-server architecture, and a 
computer program product enabling the client-server architecture, that allow a user to 
interactively select and customize a scene (i.e., update the underlying scene data) based upon 
a rendered image of the scene displayed on a client device, where the image has been 
rendered at the server level, and where image data, rather than scene data has been 
transmitted to the client device. 

Claims 14-15 and 29-30 depend, respectively, from claims 13 and 28, discussed 
above. It is therefore asserted that claims 14-15 and 29-30 are allowable for the reasons 
advanced above in support of the patentability of claims 13 and 28 and further on the basis of 
the added limitations. 

For the above reasons, it is respectfully asserted that claims 14-16 and 29-30 are 
allowable over Suits, Smirnov, French, and Horvitz. 

New Claims 31 and 32 

New claims 31 and 32 are directed, respectively, to a server and computer program 
product for enabling such a server, for use in connection with a network including at least one 
client and a communication link interconnecting the client and server. The server includes an 
image rendering module configured to render, from three-dimensional scene data representing 
a scene, a two-dimensional image, an interface configured to transmit the two-dimensional 
image over the communication link to the client, and a user interaction control module 
configured to control interactions with said at least one client in connection with rendering of 
the image from the scene data, the image rendering module being configured to render images 
from scene data representing a plurality of scenes, the user interaction control module being 
configured to select scenes for which images are to be rendered. 

As di scussed above with respect to the other pending claims, the art of record fails to 
teach or suggest the claimed combination as recited in new claims 3 1 and 32. It is therefore 
respectfully asserted that new claims 31 and 32 are allowable. 
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Conclusion : 

The Applicant believes the claims to be allowable, and allowance is therefore 
respectfully requested. If any questions remain, the Examiner is cordially invited to contact 
the undersigned. 

Respectfully submitted, 
/David Jacobs/ 

David Jacobs, Reg. No. 31,770 
JACOBS & KIM LLP 
Attorneys for Applicant 
(USPTO Customer No. 45464) 



18 



